Collaborative knowledge distillation for incomplete multi-view action prediction

Predicting future actions is a key in visual understanding, surveillance, and human behavior analysis. Current methods for video-based prediction are primarily using single-view data, while the real world multiple cameras produced videos readily available, which may potentially benefit action tasks. However, it bring up new challenge: subjects more likely to be occluded by objects when captured from different angles, or suffer signal jittering transmission. To that end, this paper we propose novel student network called Collaborative Knowledge Distillation (CKD) predict with missing information under multi-view setting, i.e., incomplete prediction. First, create graph attention based teacher model capable of fusing video features task. Second, construct corruption pattern bank (CPB) simulate various segments video, each will manage one through privileged knowledge distillation. Third, account arbitrary real-world, ensemble models developed make joint The proposed framework has been extensively evaluated on popular datasets, including PKU-MMD NTU-RGB validate effectiveness our approach best not yet explored setting.